Sensitivity of Vehicle Market Share Predictions to Alternative Discrete Choice Model Specifications

When design decisions are informed by consumer choice models, uncertainty in the choice model and its share predictions creates uncertainty for the designer. We take a first step in investigating the variation in and accuracy of market share predictions by characterizing fit and forecast accuracy of multinomial logit, mixed logit, and nested logit models over a variety of utility function specifications for the US light duty new vehicle market. Using revealed preference data for years 2004-2006, we estimate a multinomial logit model for each combination of a chosen set of utility function covariates found in the literature. We then use each of the models to predict vehicle shares for the 2007 market and examine several metrics to measure fit and predictive accuracy. We find that the best models selected using any of the proposed metrics outperform random guessing yet retain substantial error in fit and prediction for individual vehicle models. For example, with no information (random guessing) 30% of share predictions are within 0.2% absolute share error in a market with an average share of ~0.4%, whereas for the best models 70% are within 0.2% (for the 2007 vehicle market this translates to an error of ~33,000 units sold). Share predictions are sensitive to the presence of utility covariates but less sensitive to the form. Models that perform well on one metric tend to perform well on the other metrics as well. In particular, models selected for best fit have comparable forecast error to those with the best forecasts, and residual error in model fit is a major source of forecast error.